342 research outputs found
Weight Space Structure and Internal Representations: a Direct Approach to Learning and Generalization in Multilayer Neural Network
We analytically derive the geometrical structure of the weight space in
multilayer neural networks (MLN), in terms of the volumes of couplings
associated to the internal representations of the training set. Focusing on the
parity and committee machines, we deduce their learning and generalization
capabilities both reinterpreting some known properties and finding new exact
results. The relationship between our approach and information theory as well
as the Mitchison--Durbin calculation is established. Our results are exact in
the limit of a large number of hidden units, showing that MLN are a class of
exactly solvable models with a simple interpretation of replica symmetry
breaking.Comment: 12 pages, 1 compressed ps figure (uufile), RevTeX fil
Relationship between clustering and algorithmic phase transitions in the random k-XORSAT model and its NP-complete extensions
We study the performances of stochastic heuristic search algorithms on
Uniquely Extendible Constraint Satisfaction Problems with random inputs. We
show that, for any heuristic preserving the Poissonian nature of the underlying
instance, the (heuristic-dependent) largest ratio of constraints per
variables for which a search algorithm is likely to find solutions is smaller
than the critical ratio above which solutions are clustered and
highly correlated. In addition we show that the clustering ratio can be reached
when the number k of variables per constraints goes to infinity by the
so-called Generalized Unit Clause heuristic.Comment: 15 pages, 4 figures, Proceedings of the International Workshop on
Statistical-Mechanical Informatics, September 16-19, 2007, Kyoto, Japan; some
imprecisions in the previous version have been correcte
Slow nucleic acid unzipping kinetics from sequence-defined barriers
Recent experiments on unzipping of RNA helix-loop structures by force have
shown that about 40-base molecules can undergo kinetic transitions between two
well-defined `open' and `closed' states, on a timescale = 1 sec [Liphardt et
al., Science 297, 733-737 (2001)]. Using a simple dynamical model, we show that
these phenomena result from the slow kinetics of crossing large free energy
barriers which separate the open and closed conformations. The dependence of
barriers on sequence along the helix, and on the size of the loop(s) is
analyzed. Some DNAs and RNAs sequences that could show dynamics on different
time scales, or three(or more)-state unzipping, are proposed.Comment: 8 pages Revtex, including 4 figure
Trajectories in phase diagrams, growth processes and computational complexity: how search algorithms solve the 3-Satisfiability problem
Most decision and optimization problems encountered in practice fall into one
of two categories with respect to any particular solving method or algorithm:
either the problem is solved quickly (easy) or else demands an impractically
long computational effort (hard). Recent investigations on model classes of
problems have shown that some global parameters, such as the ratio between the
constraints to be satisfied and the adjustable variables, are good predictors
of problem hardness and, moreover, have an effect analogous to thermodynamical
parameters, e.g. temperature, in predicting phases in condensed matter physics
[Monasson et al., Nature 400 (1999) 133-137]. Here we show that changes in the
values of such parameters can be tracked during a run of the algorithm defining
a trajectory through the parameter space. Focusing on 3-Satisfiability, a
recognized representative of hard problems, we analyze trajectories generated
by search algorithms using growth processes statistical physics. These
trajectories can cross well defined phases, corresponding to domains of easy or
hard instances, and allow to successfully predict the times of resolution.Comment: Revtex file + 4 eps figure
Theory of spike timing based neural classifiers
We study the computational capacity of a model neuron, the Tempotron, which
classifies sequences of spikes by linear-threshold operations. We use
statistical mechanics and extreme value theory to derive the capacity of the
system in random classification tasks. In contrast to its static analog, the
Perceptron, the Tempotron's solutions space consists of a large number of small
clusters of weight vectors. The capacity of the system per synapse is finite in
the large size limit and weakly diverges with the stimulus duration relative to
the membrane and synaptic time constants.Comment: 4 page, 4 figures, Accepted to Physical Review Letters on 19th Oct.
201
Tricritical Points in Random Combinatorics: the (2+p)-SAT case
The (2+p)-Satisfiability (SAT) problem interpolates between different classes
of complexity theory and is believed to be of basic interest in understanding
the onset of typical case complexity in random combinatorics. In this paper, a
tricritical point in the phase diagram of the random -SAT problem is
analytically computed using the replica approach and found to lie in the range
. These bounds on are in agreement with previous
numerical simulations and rigorous results.Comment: 7 pages, 1 figure, RevTeX, to appear in J.Phys.
- …